Electronic apparatus, controlling method of electronic apparatus and server

ABSTRACT

An electronic apparatus which registers a device to a server by using a voice, and a method therefor are provided. The electronic apparatus includes a communication circuit, a microphone, a memory for storing computer executable instructions, and at least one processor configured to execute the computer executable instructions to acquire, from a voice received through the microphone, information on an external device which a user wishes to register, based on an external device corresponding to the acquired information being searched through the communication circuit, control the communication circuit to transmit information on an access point to the external device to enable the external device to communicate with a server, and control the communication circuit to transmit a registration request with respect to the external device to the server.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. § 119of a Korean patent application number 10-2018-0019198, filed on Feb. 19,2018, in the Korean Intellectual Property Office, the disclosure ofwhich is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to an electronic apparatus, a controlling methodof the electronic apparatus, and a server. More particularly, thedisclosure relates to an electronic device which registers a device to aserver with a voice, a controlling method of the electronic apparatus,and a server.

2. Description of Related Art

Currently, along with the development of computer technology,communication technology, and home electronics technology, a networkmanagement service in which home and factory devices and systems areinterconnected with network has emerged and is getting the spotlight asthe future-oriented technology.

All devices in a home or factory connected by a network are capable ofmutually transmitting data and can communicate with various electronicapparatus. For example, a user can control all home appliances in a homeregardless of time and place by using a user interface (UI) provided inan electronic apparatus such as a smart phone.

In order to use this service, a device needs to be set up first and theset-up of a device requires a procedure to register a device to aserver.

A device is registered using an application of a smart phone accordingto the related art. Therefore, a smartphone is essential for using thedevice, and all users have to install the application on their ownsmartphone. In addition, in order to register a plurality of devices,there has been a problem that a user has to manually register eachdevice sequentially.

As such, a method of registering each device by utilizing an applicationof a smartphone requires a long time, and there is a difficulty for theelderly to use this method. Also, since anyone can use the device byusing the smartphone in which the application is installed, there is aproblem that the device could be used by others.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providean electronic apparatus capable of registering a device to a servereasily using a voice, a controlling method of the electronic apparatus,and a server.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, an electronic apparatusis provided. The electronic apparatus includes a communication circuit,a microphone, a memory for storing computer executable instructions, andat least one processor configured to execute the computer executableinstructions to acquire, from a voice received through the microphone,information on an external device which a user wishes to register, basedon an external device corresponding to the acquired information beingsearched through the communication circuit, control the communicationcircuit to transmit information on an access point (AP) to the externaldevice to enable the external device to communicate with a server, andcontrol the communication circuit to transmit a registration requestwith respect to the external device to the server.

In accordance with another aspect of the disclosure, a server isprovided. The server includes a communication circuit comprisingcircuitry, a memory for storing computer executable instructions, and atleast one processor configured to execute the computer executableinstructions to receive a voice from an electronic apparatus through thecommunication circuit, based on the voice being related to registrationof a first device, control the communication circuit to transmitinformation on the first device to the electronic apparatus, and basedon a registration request for the first device being received from theelectronic apparatus, perform a registration procedure for the firstdevice.

In accordance with another aspect of the disclosure, a controllingmethod of an electronic apparatus is provided. The controlling method ofan electronic apparatus includes acquiring, from a voice receivedthrough the microphone, information on an external device which a userwishes to register, and based on an external device corresponding to theacquired information being searched, transmitting information on an APto the external device to enable the external device to communicate witha server, and transmitting a registration request with respect to theexternal device to the server.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a view to describe a system according to an embodiment of thedisclosure;

FIG. 2 is a block diagram to describe a configuration of an electronicapparatus according to an embodiment of the disclosure;

FIG. 3 is a flowchart to describe a procedure to register a device to aserver according to an embodiment of the disclosure;

FIG. 4 is a view to describe a method for registering a device with avoice according to an embodiment of the disclosure;

FIG. 5 is a flowchart to describe a procedure to register a plurality ofdevices at a time according to an embodiment of the disclosure;

FIG. 6 is a block diagram to describe a configuration of a serveraccording to an embodiment of the disclosure;

FIG. 7 is views to describe an embodiment to register devices by useraccounts to a server according to an embodiment of the disclosure;

FIG. 8 is views to describe another embodiment to register devices byuser accounts to a server according to an embodiment of the disclosure;

FIG. 9 is a view to describe an embodiment to cancel an exclusive use ofthe device according to an embodiment of the disclosure;

FIG. 10 is a view to describe an embodiment to request confirmation froma user to register a device according to an embodiment of thedisclosure;

FIG. 11 is a view to describe a user terminal device according to anembodiment to communicate with a server according to an embodiment ofthe disclosure; and

FIG. 12 is a flowchart to describe a controlling method of an electronicapparatus according to an embodiment of the disclosure.

Throughout the drawings, it should be noted that like reference numbersare used to depict the same or similar elements, features, andstructures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding of variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

First, the terms used in the Specification and claims have chosengeneric terms in view of their function in various embodiments of thedisclosure. However, these terms may vary depending on the intentions,legal or technical interpretations of the artisan skilled in the art,and the emergence of new technologies. In addition, some terms may beterms arbitrarily selected by the applicant. These terms may beconstrued in the meaning defined herein and, unless otherwise specified,may be construed on the basis of the entire contents of thisSpecification and common technical knowledge in the art.

In the following description, like drawing reference numerals are usedfor like elements, even in different drawings. The matters defined inthe description, such as detailed construction and elements, areprovided to assist in a comprehensive understanding of the embodiments.However, it is apparent that the embodiments may be practiced withoutthose specifically defined matters. Also, well-known functions orconstructions are not described in detail since they would obscure thedescription with unnecessary detail.

The terms such as “first,” “second,” and so on may be used to describe avariety of elements, but the elements should not be limited by theseterms. The terms are used only for the purpose of distinguishing oneelement from another.

A singular expression includes a plural expression, unless otherwisespecified. It is to be understood that the terms such as “comprise” or“consist of” are used herein to designate a presence of characteristic,number, operation, element, component, or a combination thereof, and notto preclude a presence or a possibility of adding one or more of othercharacteristics, numbers, operations, elements, components or acombination thereof.

In the example embodiments of the disclosure, a ‘module’ or a ‘unit’ mayperform at least one function or operation, and be implemented ashardware (e.g., circuitry) or software, or as a combination of hardwareand software. Further, except for the ‘module’ or the ‘unit’ that has tobe implemented as particular hardware (e.g., a dedicated processor), aplurality of ‘modules’ or a plurality of ‘units’ may be integrated intoat least one module and implemented as at least one processor.

Further, in the embodiment of the disclosure, when a part is connectedto another part, this includes not only a direct connection but also anindirect connection through another medium. Also, the meaning that apart includes an element does not exclude other elements, but mayinclude other elements, unless specifically stated otherwise.

The disclosure will be described in greater detail below with referenceto the accompanying drawings.

The electronic apparatus according to various embodiments may include atleast one of a smartphone, a tablet personal computer (PC), a mobilephone, a video phone, an e-book reader, a desktop PC, a laptop computer,a netbook computer, a workstation, a server, a personal digitalassistant (PDA), a portable multimedia player (PMP), an moving pictureexperts group phase 1 or phase 2 (MPEG-1 or MPEG-2) audio layer-3 (MP3)player, a mobile medical device, a camera, or a wearable device.According to various embodiments, the wearable device may be of the typeof accessory (e.g., a watch, a ring, a bracelet, a bracelet, a necklace,glasses, a contact lens or a head-mounted-device (HMD)), a fabric orclothing (e.g., an electronic garment), a body attachment (e.g., a skinpad or tattoo), or a bio-implantable (e.g., implantable circuit).

In another embodiment, the electronic apparatus may be a home appliance.Home appliances may include at least one of, for example, televisions(TVs), digital video disc (DVD) players, audio, refrigerators,air-conditioners, vacuum cleaners, ovens, microwaves, washing machines,air cleaners, set-top box, home automation control panel, a securitycontrol panel, a TV box such as Samsung HomeSync™, Apple TV™ or GoogleTV™), a game console (e.g., Xbox™, PlayStation™), an e-dictionary, anelectronic key, a camcorder, or an electronic photo frame.

In another embodiment, the electronic apparatus 100 (e.g., first device100) may include at least one of a variety of medical devices such asvarious portable medical measurement devices (a blood glucose meter, aheart rate meter, a blood pressure meter or a temperature measuringdevice, a magnetic resonance angiography (MRA), a magnetic resonanceimaging (MRI), computed tomography (CT), a photographing device, or anultrasonic device, etc.), a navigator, global navigation satellitesystem (GNSS), an event data recorder (EDR), a flight data recorder(FDR), automotive infotainment devices, marine electronic equipment(e.g., marine navigation devices, gyro compasses, etc.), avionics,security devices, head units for the vehicle, industrial or home robots,automatic teller's machine (ATM), point of sales (POS) of a store, orinternet of things (e.g., light bulbs, various sensors, electricity orgas meters, sprinkler devices, fire alarms, thermostats, street lights,a toaster, a fitness equipment, a hot water tank, a heater, a boiler,etc.).

In another embodiment, the electronic apparatus 100 may include at leastone of a piece of furniture or a part of a building/structure, anelectronic board, an electronic signature receiving device, a projector,or various measuring devices (e.g., water, electricity, gas, or radiowave measuring instruments, etc.). In various embodiments, theelectronic apparatus may be a combination of one or more of the variousdevices described above. The electronic apparatus according to someembodiments may be a flexible electronic apparatus. Further, theelectronic apparatus according to the embodiment of the disclosure isnot limited to the above-described devices, and may include a newelectronic apparatus according to technological advancement.

FIG. 1 is a view to describe a system 1000 according to an embodiment ofthe disclosure.

Referring to FIG. 1 , a system 1000 includes a first device 100, asecond device 200, a server 300, and an access point (AP) 400.

The first device 100 is an electronic apparatus and may be a homeappliance such as a TV, an air-conditioner, a refrigerator, a washingmachine, a robot cleaner, a humidifier, a door lock, a dish washer, asecurity camera, a closed circuit television (CCTV), a security sensor,light emitting diode (LED), a lamp, a heating device, a power measuringdevice, a power socket, an electricity socket, a multi-tap, personalcomputer (PC), internet protocol (IP) camera, internet telephone,wired/wireless telephone, electrically-controllable curtain, and blind.

The first device 100 may communicate with an external device such as thesecond device 200 and the server 300. For example, the first device cancommunicate with an external device by communication methods such asWi-Fi, ZigBee, and Z-Wave.

The second device 200 is a device that can register the first device 100on the server 300 based on the user's voice. The second device 200 maybe a device pre-registered in the server 300. The registration of thesecond device 200 can be performed through a third device, for example,a user terminal device 500 shown in FIG. 11 .

The second device 200 may control the first device 100 based on a user'svoice.

The second device 200 is an electronic apparatus which can be operatingby a voice command and includes a microphone. The second device 200 maybe an apparatus, for example, a smartphone, a tablet, and a digital TV,or an artificial intelligent speaker that can perform an action inresponse to a user command as if the speaker has a conversation with auser.

The second device 200 may interact with the first device 100 through theserver 300 or by using device-to-device (D2D) or peer-to-peer (P2P)connection.

The second device 200 may execute a smart home application and providevarious services to the user through its own display. The smart homeapplication may be embedded in the second device 200 or downloaded froman app store such as Google Play™ or Samsung Apps™. The first device 100can be registered or controlled in the server 300 by executing the smarthome application in the second device 200.

The server 300 may be implemented as a cloud server. According to oneembodiment, the server 300 may include an account server and a serviceserver. The account server can perform user authentication. Inparticular, the account server can perform user authentication based onvoice recognition. The service server can perform functions such as webportal, status information management of devices, remote access, deviceupdate, message notification, and so on.

A service provided by the system 1000 can be largely divided into deviceregistration and device control.

Device registration is a procedure, for example, that the second device200 supports a new first device 100 to access the access point (AP) 400,and registers the first device 100 to the server 300.

Device control is a procedure that a user controls the first device 100registered in the server 300 through the server 300 using the seconddevice 200 or a device different from the second device 200.

When the first device 100 and the second device 200 are used at home,they can be called a home device. The first device 100, the seconddevice 200 and the server 300 can communicate directly or through thehome gateway. The home gateway can manage and control the home devices,in overall, in the home network including the first device 100 and thesecond device 200 in cooperation with the server 300.

FIG. 2 is a block diagram to describe a configuration of an electronicapparatus according to an embodiment of the disclosure.

Referring to FIG. 2 , the second device 200 includes a microphone 210, acommunicator 220 (e.g., a communication circuit or a transceiver), amemory 230, and a processor 240 (e.g., at least one processor).

The microphone 210 is a structure for receiving sound. The microphone130 may convert the received sound into an electrical signal. Themicrophone 210 may be implemented integrally with or separated from thesecond device 200. The separated microphone 210 may be electricallyconnected to the second device 200.

The communicator 220 may include various communication circuitry toperform communication with various types of external devices. Thecommunicator 220 may communicate with an external device using at leastone of Wi-Fi, Bluetooth, near field communication (NFC), infrared dataassociation (IrDA), radio frequency identification (RFID), ultra-wideband (UWB), Wi-Fi direct, Z-wave, internet protocol version 4 (IPv4)over low-power wireless personal area networks (4LoWPAN) or 6LoWPAN,general packet radio services (GPRS), Weightless, digital living networkalliance (DLNA), ANT+, digital enhanced cordless telecommunications(DECT), wireless local area network (WLAN), global system for mobilecommunications (GSM), universal mobile telecommunication system (UMTS),and wireless broadband (WiBRO). The communicator 220 may be implementedas a communication chip and a transceiver to perform communication asdescribed above.

The memory 230 may, for example, include an embedded memory 232 or anexternal memory 234. The embedded memory 232 may, for example, includeat least one of a volatile memory (e.g. dynamic random access memory(DRAM)), a static RAM, a synchronous dynamic RAM (SDRAM), a non-volatilememory (e.g. one time programmable read only memory (OTPROM)), aprogrammable ROM (PROM), an erasable and programmable ROM (EPROM), anelectrically erasable and programmable ROM (EEPROM), a mask ROM, a flashROM, a flash memory (e.g. NAND flash or NOR flash, etc.), a hard drive,and a solid state drive (SSD).

The external memory may further include a flash drive, such as a compactflash (CF), a secure digital (SD), a micro secure digital (Micro-SD), amini secure digital (Mini-SD), an extreme digital (xD), a multi-mediacard (MMC), or a memory stick. The external memory may be connected tothe electronic apparatus 201 functionally and/or physically throughvarious interfaces.

The memory 230 may be implemented as not only a storage medium insidethe second device 200 but also a web server through network, and so on.

The memory 23 is accessed by the processor 240, andread/write/modify/delete/update of data by the processor 240 can beperformed. The term memory in this document refers to the memory 230(not shown), a ROM in the processor 240, a RAM, or a memory card (forexample, a micro SD card and a memory stick) provided on the seconddevice 200. The memory 230 may store computer executable instructions.

The processor 240 is a configuration for controlling the overalloperation of the second device 200. The processor 240 may beimplemented, for example, as a central processing unit (CPU), anapplication specific integrated circuit (ASIC), a system on chip (SoC),a MICOM, or the like. The processor 240 may operate an operating systemor an application program to control a plurality of hardware or softwarecomponents connected to the processor 240, and may perform various dataprocessing and operations. According to one embodiment, the processor240 may further include a graphics processing unit (GPU) and/or an imagesignal processor. The processor 240 may execute the computer executableinstructions to perform the functions of the second device 200 accordingto various embodiments of the disclosure.

For example, the processor 240 may acquire information on an externaldevice that the user desires to register from the voice received throughthe microphone 210, and when an external device corresponding to theacquired information is searched through the communicator 220 from amongthe external devices, the processor may control the communicator 220 totransmit, to the external device, information on the access point 400 sothat the external device can communicate with the server 300, andcontrol the communicator 220 to transmit a request for registration withrespect to the external device to the server 300.

Here, the processor 240 may search for an external device whose serviceset identifier (SSID) includes the acquired information on the externaldevice.

Information on an access point 400 which the second device 200 transmitsto an external device may be information used by the second device 200to contact the access point 400.

According to one embodiment, the processor 240 uses the learned speechrecognition model using the artificial intelligence (AI) algorithmstored in the memory 230 and may acquire information on an externaldevice which the user wishes to register from voice received through themicrophone 210.

According to still another embodiment, the server 300 may have a learnedspeech recognition model using an artificial intelligence algorithm, andthe processor 240 may control the communicator 220 to transmit the voicereceived through the microphone 210 to the server 300, and receiveinformation on an external device from the server 300 through thecommunicator 220.

The processor 240, when information on a plurality of external deviceswhich a user desires to register is acquired from the voice receivedthrough the microphone 210, may control the communicator 220 to transmitinformation on the access point sequentially to the plurality ofexternal devices and control the communicator 220 to transmit aregistration request for the plurality of external devices to the server300.

According to an embodiment, the second device 200 further includes aspeaker, and when the information about the external device to which theuser wants to register is acquired from the voice, the processor 240controls the communicator 220 to transmit control information forcausing the external device to output information indicating that theexternal device is to be registered, outputs an inquiry voice toregister the external device through a speaker, and when voice to agreethe inquiry voice is received through the microphone 210, controls thecommunicator 220 to transmit the information on the access point to theexternal device, and controls the communicator 220 to transmit aregistration request with respect to the external device to the server300.

According to an embodiment, the processor 240, when audio data includinga voice indicating that registration of the external device is completedis received from the server 300, may control a speaker of the seconddevice 200 to output the audio data.

FIG. 3 is a view to describe a procedure to register the first deviceaccording to an embodiment to the server 300. Here, the second device200 may be a device previously registered in the server 300.

Referring to FIG. 3 , the user speaks to register the first device 100,and the user's voice is received through a microphone 210 of the seconddevice 200 in operation S310. For example, if the first device 100 is arefrigerator, the user can say “register refrigerator.” Alternatively,the user may include a trigger word to activate a voice command functionof the first device 100. For example, a user can say “Bixby, register arefrigerator.”

The second device 200 acquires information about a device to beregistered by the user from the received voice in operation S320. Forthis, an analysis process for voice can be performed. For example, thesecond device 200 may transmit the received voice to the server 300, andthe server 300 may include a speech recognition model and may obtaincontext information from the voice received using the speech recognitionmodel. The speech recognition model may be an artificial intelligencemodel learned by artificial intelligence algorithms. For example, thespeech recognition model may include a Speech to Text (SPP) module forconverting speech to text, and a Natural Language Understanding (NLU)module for processing natural language understanding of the convertedtext. The server 300 may transmit the context information of the voiceto the second device 200. The context information may includeinformation on the first device and information indicating that thefirst device is to be registered. Here, the information about the firstdevice may include the name of the first device. The server 300 may usea word which a user speaks as it is or use a modified word as a name ofthe first device. For example, even if the user says “register a TV,”the server 300 can use “TV” rather than “television” as a name of thedevice. The name of the first device may be provided in a languagecorresponding to the user's voice. According to one embodiment, if thelanguage corresponding to the user's voice is not English, the server300 may provide a translation of the first device name into English.

As described above, the second device 200 can acquire information of adevice to be registered through the server 300. According to stillanother embodiment, the second device 200 includes a speech recognitionmodel, and the second device 200 can acquire information on the devicethat the user intends to register from the voice.

The second device 200 can communicate with the first device 100 based onthe information about a device which the user wishes to register that isacquired by the user from the voice in operation S330. The connectionbetween the second device 200 and the first device 100 will be describedbelow.

According to an embodiment, the first device 100 may execute a soft APfunction, allow access of the second device 200 as if the AP to connectcommunication with the second device 200.

Specifically, the first device 100 may transmit a beacon periodically byexecuting a soft AP function. The beacon is a signal for allowing thefirst device 100 to inform its surroundings of its existence so that aperipheral device can detect the first device and participate in themutual communication. The beacon may include a Service Set Identifier(SSID), a beacon interval, capability information, and the like of thefirst device 100 and may be variously expressed using a vendor specificfield.

The soft AP function can be executed manually or automatically.According to the embodiment that is manually performed, for example, thesoft AP function can be executed by pressing a button or the likeprovided on the first device 100 by the user. According to theembodiment that is automatically performed, when the first device 100 isnot registered in the server 300, the first device 100 can automaticallyexecute the soft AP function. For example, when the user purchases thefirst device 100 and connects the power source, the first device 100 canautomatically execute the soft AP function. According to still anotherembodiment, it is also possible for the second device 200 to execute theAP function of the first device 100. For example, when the user utters avoice to register a specific device, the second device 200 can transmita signal (IR signal, ultrasonic signal, etc.) for executing the APfunction. The first device 100 includes a configuration capable ofreceiving a signal transmitted from the second device, and execute theAP function by receiving the signal.

According to one embodiment of the disclosure, the SSID contained in thebeacon transmitted by the first device 100 may include a characterstring that allows the first device to recognize what the first deviceis, and can be uniquely assigned to each device or by devices of thesame model. If necessary, the SSIDs can be dynamically allocated, and allocated using a specific rule through the application. When the firstdevice 100 is, for example, a smart home device (a device supporting theIoT function), the SSID can be generated based on a smart home protocol(SHP), which is an indicator indicating that the smart home device,types of a device, a production date, a version and so on. For example,if the first device 100 is a refrigerator that supports smart homefunctionality and is produced in 2014, the SSID 501 of the first device100 may be configured as “SHP.Fridge.2014.”

When the beacon is received, the second device 200 may analyzeinformation included in the beacon and determine whether the firstdevice 100 transmitting the beacon corresponds to the device that thesecond device 200 desires to register. For example, the second device200 may acquire information about a device to be registered by the userfrom the voice, check whether the beacon includes the informationcorresponding to the acquired information, and determine whether thefirst device 100 corresponds to the device which the first device 100would like to register. For example, if the information about the deviceobtained from the voice includes the word “Fridge”, it can be determinedthat the first device 100 whose SSID “SHP.Fridge.2014” includes “Fridge”is a device which a user wishes to register.

If it is determined that the first device 100 transmitting the beacon isa device which the user wishes to register, the second device 200 maycommunicate with the first device 10 using information included in thebeacon.

According to still another embodiment of the disclosure, when the seconddevice 200 acquires information on a device to be registered by a userfrom a voice, the second device may broadcast a probe request, and thefirst device 100 which detects the probe request can transmit the proberesponse to the second device 200. The probe response may includeinformation such as an SSID as the beacon described above. In the samemanner as the embodiment in which the beacon is used, the second device200 may analyze the information included in the probe response anddetermine whether or not the first device 100 that has transmitted theprobe response corresponds to the device that the user intends toregister. If it is determined that the first device 100 transmitting theprobe response corresponds to the device to be registered, the seconddevice 200 may communicate with the first device 100 using theinformation included in the probe response.

According to still another embodiment, the first device 100 and thesecond device 200 can be communicated via Wi-Fi P2P.

Specifically, when the first device 100 and the second device 200 enterthe P2P terminal discovery mode, they can perform the P2P terminalsearch. For example, the first device 100 and the second device 200 maybe configured such that an application using Wi-Fi P2P is automaticallyactivated according to a user's input or need, or enter a P2P terminalsearch mode by a request of Wi-Fi simple configuration (WSC), and so on.For example, the second device 200 may enter the P2P terminal searchmode when context information including information indicating that thedevice 300 is to be registered is received from the server 300. Then,the second device 200 can transmit a signal (IR signal, ultrasonicsignal, etc.) for entering the P2P terminal search mode, and the firstdevice 100 can receive the signal and enter the P2P terminal searchmode.

The second device 200 performs a Wi-Fi P2P search to detect a devicethat the user intends to register. Here, the Wi-Fi P2P search includes adevice discovery process for recognizing each other and a servicediscovery process for a device discovered through the device searchprocess. Through such device search and service search processes, a usercan determine a device to be registered.

Specifically, in the service discovery process, it is possible to knowwhat services a counterpart device supports through a service discoveryquery and a service discovery response. For example, the serviceprotocol type field of the service discovery response includes a valuecorresponding to a service protocol type indicating which service issupported. For example, if the value 1 is Bonjour, the value 2 is UpnP,the value 3 is WS-Discovery, the value 4 is an identifier indicatingthat the device type is a refrigerator, and the value 5 is an identifierindicating that the device type is an air-conditioner, when a device hasa value of “4,” the device can be determined as a refrigerator. Theservice protocol type corresponding to the device type may be assignedto a value that is not used by another service protocol or a vendorspecific value.

The second device 200 may receive a value corresponding to a serviceprotocol type from the first device 100 and if the received valuecorresponds to the device information acquired from the voice, thesecond device may determine that the first device 100 is a device theuser wishes to register.

The second device 200 performs a Wi-Fi P2P connection with the firstdevice 100 that is determined to be a device a user wishes to register.Specifically, the first device 100 and the second device 200 may performa Provision Discovery Exchange and perform provisioning to form a Wi-FiP2P connection.

In the above description, it has been described that the first device100 and the second device 200 are connected by the Soft Ap and the Wi-FiP2P communication method. However, the first device 100 and the seconddevice 200 can be connected using various communication methods such asbluetooth low energy (BLE), Bluetooth, ZigBee, radio frequency (RE)communication, Z-wave, and so on.

When connection between the first device 100 and the second device 200is established, the second device 200 may transmit information about thepre-stored access point (AP) 400 to the first device 100 in operationS340. The information about the AP 400 may be information about the AP400 to which the second device 200 has previously been connected. Theinformation on the AP 400 may include an SSID and a security type of theAP 400, an encryption type, a security key, and the like. In addition,the second device 200 may transmit the server authentication informationto the first device 100. The server authentication information isinformation used by the first device 100 to make a registration requestto the server 300. The second device 200 may request to the server 300the server authentication information to be used by the first device 100and receive the same.

Though not illustrated in FIG. 3 , communication is established betweenthe first device 100 and the second device 200, and the first device 100may transmit identification information of the first device 100 to thesecond device 200. The identification information of the first device100 is information for allowing another device to identify the firstdevice 100 and may include information such as a type, a name, adescription, a manufacturer, a model ID, serial number, a saleslocation, versions, and so on.

The identification information of the first device 100 is temporarilystored in the second device 200 before the information of the firstdevice 100 is registered in the server 300, and after the information isregistered in the server 300, the information can be removed from thefirst device 100. The identification information of the first device 100received by the second device 200 can be used in the registrationrequest of the first device in operation S380.

Thereafter, through it is not illustrated, the first device 100 and thesecond device 200 can be disconnected. The second device 200 can beconnected to the AP 400.

The first device 100 may access the AP 400 using the information aboutthe AP 400 transmitted from the second device 200 in operation S350. Thefirst device 100 may be connected to the network by receiving an IPaddress from the AP 400 in operation S360. The first device 100 cancommunicate with an external device such as the server 300 through theAP 400. The first device 100 may send an advertisement message thatincludes information that the first device is connected to the networkto the second device 200.

The second device 200 can log in to a user account of the server 300based on the user's voice received in operation S310 (S370). Since thevoice includes information that allows the user to be identified, suchas a fingerprint, the voice can play a role such as an ID and apassword. Specifically, the second device 200 may transmit the voice ofthe user to the server 300, and when the voice feature information ofthe user pre-registered the server 300 and the feature information ofthe voice transmitted to the server 300 correspond to each other, alogin can be made. It is possible that the login may also be performedin operation S310.

The second device 200 may then transmit, to the server 300, aregistration request requesting registration of the first device 100 inoperation S380. At this time, the transmitted registration request mayinclude the identification information of the first device 100 and mayinclude information on the installation location of the second device200.

The first device 100 may transmit to the server 300 a registrationrequest using the server authentication information in operation S390.The transmitted registration request may include identificationinformation of the first device 100.

When the registration request received from the second device 200 andthe registration request received from the first device 100 correspondto each other, the server 300 may register the first device 100 to auser account logged in by voice in operation S395, Here, the server 300,if identification information of the first device included in theregistration request received from the second device 200 and the firstidentification information of the first device included in theregistration request from the first device 100 are the same from eachother, may register the first device 100 to a user account. In thiscase, the identification information of the first device 100 and theinformation on the installation location of the second device 200 may beregistered as information on the first device.

The server 300 can transmit control information to the first device 100so that information indicating the completion of the registration of thefirst device 100 is output from the first device 100. If the firstdevice 100 includes a display, a message such as “registration iscompleted” may be output via the display. When the first device 100includes a speaker, a voice such as “registration is completed” may beoutput through the speaker.

The server 300 may transmit control information to the second device 200so that information indicating completion of registration of the firstdevice 100 is output from the second device 200. If the registered firstdevice is a refrigerator and the second device 200 includes a speaker, avoice “registration of the refrigerator is completed” may be outputtedthrough the speaker, and if the second device 200 includes a display,the message “registration of the refrigerator is completed” can beoutput through the display.

According to still another embodiment of the disclosure, wheninformation about a plurality of devices to be registered by the user isobtained from the voice received through the microphone 210 of thesecond device 200, the plurality of devices can be registered in theserver 300. This will be described with reference to FIG. 5 .

FIG. 5 is a flowchart to describe a procedure to register a plurality ofdevices at a time according to an embodiment of the disclosure.

In the embodiment referring to FIG. 3 , it has been described an examplethat only one first device 100 is registered, in FIG. 5 , an example ofregistering the first device 100 and a third device 110 at the same timewill be described.

Referring to FIG. 5 , the user speaks to register the first device 100,and the voice of the user is received through the microphone 210 of thesecond device 200 in operation S510. For example, if the first device100 is a refrigerator and the third device is a TV, the user can speak“register a refrigerator and TV” as shown in FIG. 4 . Alternatively, theuser may speak including a trigger word for activating the voice commandfunction of the first device 100. For example, a user may say “Bixby,register a refrigerator and TV.”

FIG. 4 is a view to describe a method for registering a device with avoice according to an embodiment of the disclosure.

The second device 200 acquires information about a device that the userwishes to register from the received voice in operation S515.Information can be obtained from the voice as described in operationS320 of FIG. 3 . For example, the second device 200 may transmit thereceived voice to the server 300, and the server 300 may obtain thecontext information from the received voice using the speech recognitionmodel. The server 300 may transmit the context information acquired fromthe voice to the second device 200. The context information may includeinformation on the first device 100 and the third device 110 andinformation indicating that the first device 100 and the third device110 are to be registered.

As described above, the second device 200 can acquire information of adevice to be registered through the server 300. According to stillanother embodiment, the second device 200 includes a speech recognitionmodel, so that the second device 200 can acquire information on thedevice that the user wishes to register from the voice.

If the second device 200 determines that the user wants to register aplurality of devices based on the information about the device that theuser wishes to register, the second device 200 can sequentiallyestablish communication with the plurality of devices. First, the seconddevice 200 can establish a communication connection with the firstdevice 100 in operation S520. In this case, communication can beestablished as described in operation S330 of FIG. 3 .

When communication is established between the first device 100 and thesecond device 200, the second device 200 may transmit information aboutthe pre-stored access point (AP) 400 to the first device 100 inoperation S525. In addition, the second device 200 may transmit theserver authentication information to the first device 100. The firstdevice 100 may transmit the identification information of the firstdevice 100 to the second device 200. The identification information ofthe first device 100 received by the second device 200 may be used inthe registration request in operation S570.

The second device 200 may disconnect connection with the first device100 to be connected to the third device 110 in operation S530.

After the disconnection, the second device 200 may establish connectionwith the third device in operation S535. Here, as described in S330 ofFIG. 3 , communication connection can be established.

When communication is established between the first device 100 and thethird device 110, the second device 200 may transmit information aboutthe pre-stored access point (AP) 400 to the third device 110 inoperation S540. Also, the second device 200 may transmit the serverauthentication information to the third device 110. The third device 110may transmit the identification information of the third device 110 tothe second device 200. The identification information of the thirddevice 110 received by the second device 200 may be used for theregistration request in operation S570.

Then, the second device 200 may disconnect connection with the thirddevice 110 and access the AP 400.

The first device 100 may access the AP 400 using the information aboutthe AP 400 transmitted from the second device 200 in operation S545. Thefirst device 100 may be connected to the network by receiving an IPaddress from the AP 400 in operation S550. The first device 100 cancommunicate with an external device such as the server 300 through theAP 400. The first device 100 may transmit, to the second device 200, anadvertisement message that includes information that the first device isconnected to the network.

Likewise, the third device 110 can access the AP 400 using theinformation about the ALP 400 transmitted from the second device 200 inoperation S555. Then, the third device 110 is allocated with an IPaddress from the AP 400 and can be connected to the network in operationS560. The third device 110 can communicate with an external device suchas the server 300 through the AP 400. The third device 110 may transmit,to the second device 200, an advertisement message which includesinformation that the third device 110 is connected to the network.

Thereafter, the second device 200 may log in to the user account of theserver 300 based on the user's voice received in the operation S510 inoperation S565. As described in S370 of FIG. 3 , log-in can beperformed.

Thereafter, the second device 200 may transmit, to the server 300, aregistration request requesting registration of the first device 100 andthe third device 110 in operation S570. At this time, the transmittedregistration request may include the identification information of thefirst device 100 and the identification information of the third device110, and may further include information on the installation location ofthe second device 200.

The first device 100 may transmit the registration request to the server300 using the server authentication information in operation S575. Atthis time, the transmitted registration request may include theidentification information of the first device 100. The third device 110may similarly transmit the registration request to the server 300 usingthe server authentication information in operation S580. Here, thetransmitted registration request may include identification informationof the third device 110.

When the registration request received from the second device 200 andthe registration request received from the first device 100 correspondto each other, the server 300 may register the first device 100 and thethird device 100 to a user account which is logged in with voice inoperation S590. At this time, the server 300, only in a case where theidentification information of the first device included in theregistration request received from the second device 200 and theidentification information of the first device included in theregistration request received from the first device 100 are the samefrom each other, may register the first device 100 to a user account,and only in a case where the identification of the third device includedin the registration request received from the second device 200 and theidentification information of the third device included in theregistration request received from the third device 110 are the samefrom each other, may register the third device 100 to a user account. Inthis case, the identification information of the first device 100 andthe information of the installation location of the second device 200can be registered as the information of the first device, and theidentification information of the third device 110 and the informationon installation position of the second device 200 can be registered asthe information on the third device.

After that, the server 300 can transmit control information to the firstdevice 100 so that information indicating the completion of registrationof the first device 100 is output from the first device 100, andtransmit, to the third device 110, control information for causing thethird device 110 to output information informing completion ofregistration of the first device 110.

According to still another embodiment of the disclosure, the server 300may transmit control information for causing the second device 200 tooutput information indicating completion of registration of the firstdevice 100 and the third device 110 to the second device 200. In thecase where the second device 200 includes a speaker, a voice of “theregistration of the refrigerator and the TV is completed” can beoutputted through the speaker, and when the second device 200 includesthe display, “the registration of the refrigerator and TV is completed”may be output on the display.

According to an embodiment with reference to FIG. 5 , there is anadvantage that a user does not need to have an irritating process that auser needs to register each of a plurality of devices when registering aplural of devices.

FIG. 6 is a block diagram to describe a configuration of a server 300according to an embodiment of the disclosure.

The server 300 may be implemented as a cloud server, and can be dividedinto a plurality of servers according to functions. For example, theserver 300 may include an account server and a service server. Theaccount server can perform user authentication. In particular, theaccount server can perform user authentication based on voicerecognition. The service server can perform functions such as webportal, information management for smart home devices, service interfacefor 3rd party application, remote access, device update, messagenotification, and so on.

Referring to FIG. 6 , the server includes a communicator 310 (e.g., acommunication circuit or a transceiver), the memory 320, and a processor330 (e.g., at least one processor).

The communicator 310 (e.g., a communication circuit or transceiver) mayinclude various communication circuitry for performing communicationwith various types of external devices. The communicator 310 maycommunicate with an external device using at least one of Wi-Fi,Bluetooth, near field communication (NFC), infrared data association(IrDA), radio frequency identification (RFID), ultra-wideband (UWB),Wi-Fi Direct, Z-wave, 4LoWPAN, GPRS, Weightless, Digital Living NetworkAlliance (DLNA), ANT+, Digital Enhanced Cordless Telecommunications(DECT), Wireless Local Area Network (WLAN), Global System for Mobilecommunications (GSM), Universal Mobile Telecommunication System (UNITS)wireless broadband (WiBRO), and the like. The communicator 310 may beimplemented as a communication chip and a transceiver for performing thecommunication method described above.

A memory 320 may include, for example, internal memory or externalmemory. The internal memory may include at least one of a volatilememory such as a dynamic RAM (DRAM), a static RAM (SRAM), or asynchronous dynamic RAM (SDRAM); and a non-volatile memory such as anone-time programmable ROM (OTPROM), programmable ROM (PROM), erasableand programmable ROM (EPROM), electrically erasable and programmable ROM(EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NORflash), hard drive, or solid state drive (SSD).

The external memory may be a flash drive, for example, a compact flash(CF), a secure digital (SD), a micro secure digital (Micro-SD), minisecure digital (Mini-SD), extreme digital (xD), multi-media card (MMC),or a memory stick. The external memory may be functionally and/orphysically connected to the server 300 via various interfaces.

The memory 320 is accessed by the processor 330, and dataread/write/modify/delete/update by the processor 330 can be performed.The term memory in the disclosure may include the memory 320, a ROM inprocessor 330, a RAM, or a memory card (e.g., micro SD card, memorystick) mounted in the server 300. The memory 320 may store computerexecutable instructions.

The processor 330 is a configuration for controlling the overalloperation of the server 300. The processor 330 may be implemented as,for example, a CPU, ASIC, SoC, MICOM, or the like. The processor 330 mayoperate an operating system or an application program to control aplurality of hardware or software components connected to the processor330, and may perform various data processing and operations. Accordingto one embodiment, the processor 330 may further include a graphicsprocessing unit (GPU) and/or an image signal processor. The processor330 may execute the computer-executable instructions to perform thefunctions of the server 300 according to various embodiments of thedisclosure.

The user can generate an own account in the server 300 using a PC, asmart phone, or the like. For example, a user terminal device 500 shownin FIG. 11 can be used. The user can access the web page for accountgeneration through the user terminal device 500 or execute theapplication for generating a user account. A web page or an applicationprovides a UI screen for a new subscription, and a user can create anaccount through inputting an ID, a password, and the like to be used onthe UI screen. The user can input the ID and password at a later timeand log in to his or her account of the server 300 or log in usinganother way.

In particular, according to one embodiment of the disclosure, a user maylog in to his or her account of the server 300 by voice. In order to usethe voice log-in function, for example, the user enters an ID and apassword into the user terminal device 500 to log into his or heraccount of the server 300 and speaks to a microphone provided in theuser terminal device 500, and then the user's voice can be transmittedto the server 300. In this case, the user's smartphone may be requiredto make a specific statement to obtain the feature information of theuser's voice to be used for log-in. For example, the message “This is avoice registration procedure for using the voice log-in method. Pleasesay the following words in order” may be displayed on the user terminaldevice 500. Then, when the user speaks, the user terminal device 500 maytransmit the user's voice to the server 300, the server 300 may acquirethe voice feature information from the received voice, and may store thevoice feature information in the memory 320 as the voice log-ininformation. The user can log in to the own account in the server 300only by speaking through the microphone provided in the user terminaldevice 500 or the second device 200. Specifically, when the server 300receives voice from the user terminal device 500 or the second device200, the server 300 may acquire the voice feature information from thereceived voice, and when the acquired voice feature informationcorresponds to the voice feature information of a user pre-registered inthe memory 320, log-in to the corresponding user account can beperformed.

The server 300 may perform a voice recognition function using artificialintelligence (AI) technology. For example, the speech recognition modelwhich is learned using the AI algorithm may be stored in the server 300.

When voice (audio data) is received from the second device 200 throughthe communicator 310, the processor 330 may determine whether the voiceis of a registered user, and acquire context information from the voiceusing the speech recognition model. For example, if the received voiceis “register the refrigerator”, it can be determined that the context ofthe voice means registration of the refrigerator.

If the voice received from the second device 200 through thecommunicator 310 is related to the registration of the first device 100,the processor 330 may control the communicator 310 to transmitinformation regarding the first device 100 to the second device 200. Theinformation about the first device 100 may include a name of the firstdevice 100. The information about the first device 100 transmitted tothe second device 200 is information for allowing the second device 200to identify the first device 100 among the scanned peripheral devices,and may include information corresponding to the SSID of the firstdevice 100, or information corresponding to the service protocol.

Thereafter, when a registration request for the first device 100 isreceived from the second device 200, the processor 330 may perform theregistration procedure for the first device 100. An account by users ismanaged in the server 300, and the first device 100 can be registered ina specific user account. For example, as shown in FIG. 7 , a device maybe registered for each user. The first device 100, the third device 120and the fourth device 130 are registered in the account of the firstuser and the fifth device 140 and the sixth device 150 are registered inthe account of the second user. The devices registered in the server 300have unique IDs, and the devices registered in the account of one usercan have the same group ID. The unique ID and group ID can be used fordevice control and management. The devices can store a unique ID, agroup ID, and can be used for connection to the server 300.

FIG. 7 is views to describe an embodiment to register devices by useraccounts to a server according to an embodiment of the disclosure.

The processor 330 may, if the voice received from the second device 200is related to registration of a plurality of devices, control thecommunicator 310 to transmit information on the plurality of devices tothe second device 200.

According to an embodiment, it is possible to register a device to auser account based on a voice.

For example, when the voice received from the second device 200corresponds to the voice of the first user registered in the processor330 and the registration request for the first device is received fromthe second device 200, the processor 330 may register the first deviceto the account of a first user. That is, if the voice is a voice of thefirst user, the device can be registered to an account of the firstuser, and if the voice is a voice of the second user, the device can beregistered to an account of the second user.

The memory 320 of the server 300 may store the voice feature informationof the first user and the voice feature information of the second userand the processor 330 may, if the received voice is for registration ofa device, compare the received voice feature information with voicefeature information of pre-registered users to identify a usercorresponding to the voice, and register a device to an account of theidentified user.

According to an embodiment, the devices registered to an account of thefirst user can be set so that the first user can exclusively use, anddevices registered to an account of the second user can be set so thatthe second user can exclusively use. This setting can be done at theregistration stage, or after registration in accordance with a user'smanipulation.

If the first device 100 of the first user is set to use the first userexclusively, information may be required to authenticate the first userto use the first device 100. For example, when the first device 100 isto be used, the first device 100 may request biometric information(voice, face, fingerprint, iris, etc.), password, pattern, etc. of thefirst user as authentication information. Only in the case where theauthentication information is input, the first device 100 can be used.

According to an embodiment, a device can be registered as a commondevice. The device which is registered for a common use can be used byusers of a predetermined group.

FIG. 8 is a view to describe this embodiment to register devices by useraccounts to a server according to an embodiment of the disclosure.

Referring to FIG. 8 , when the first user A says “registerair-conditioner as a common device”, the voice is input through themicrophone 210 of the second device 200, and can be transmitted to theserver 300. The server 300 can recognize that the voice is the voice ofthe first user A, and may, from among devices registered to an accountof the first user A, register the air-conditioner 130 to an account ofthe second user who belongs to a same group as the first user.Accordingly, not only the first user but also the second user can usethe air-conditioner 130.

In the above-described embodiment, it has been described that thedevices already registered in the accounts of the first user A areregistered as common devices, but it is also possible to register a newdevice not registered in any user's account as a common device. Forexample, when the user wishes to newly register the air-conditioner, ifthe user speaks “register the air-conditioner as a common device,” thevoice can be inputted through the microphone 210 of the second device200 and transmitted to the server 300. The server 300 may analyze thevoice and transmit, to the second device 200, the information indicatingthat the information on the air-conditioner and the air-conditioner areto be registered, and the second device 200 may perform a registrationprocedure as described above with reference to FIG. 3 , and the seconddevice 200 may transmit, to the server 300, a registration requestrequesting registration of the air-conditioner as a common device. Theserver 300, when a registration request is received, may register theair-conditioner to an account of the first user and an account of thesecond user belonging to the same group as the first user.

As in the above-described embodiments, devices may be registered inaccounts of all users belonging to the same group, or according to stillanother example, a user who may use the device together may bedesignated and registered. For example, if the first user says “registerthe air-conditioner to be used along with Jane (second user)”, theair-conditioner can be registered in the accounts of the first user andthe second user.

According to the embodiments described above, since the account can bemanaged by the voice recognition of the user and the user other than theregistered user cannot use it, the privacy management can be performed.For example, by simply saying “register TV as my device,” it is possibleto register the device as a personal device, and by only simply“register TV as a common device,” it is possible to register the deviceas a common device. Therefore, even users who are not familiar with theIoT device of the related art can register the device easily.

According to one embodiment of the disclosure, the first device 100registered in the account of the first user can be set such that thefirst user can exclusively use, and a device registered to an account ofthe second user can be set so that only the second user can exclusivelyuse. For example, the first device 100 can be used only whenpre-registered authentication information is input. The authenticationmethod may be password input, voice recognition, face recognition, andthe like. It is also possible for the user to directly input informationfor authentication for cancelling exclusive use to the first device 100and to input the information for authentication through the seconddevice 200 or another device. For example, when the voice of the firstuser A desiring to cancel the exclusive use of the first device 100through the microphone 210 of the second device 200 is received, thesecond device 200 may transmit the voice to the server 300, and theprocessor 330 may control the communicator 310 to transmit the controlinformation for cancelling exclusive use of the first device 100 to thefirst device 100.

The voice of the first user A to cancel the exclusive use can be a voiceto register the first device 100 as a common device as described above.

According to still another embodiment, the voice of the first user whowishes to cancel the exclusive use may be a voice to temporarily permitthe use of the first device 100 to another user. In this case, theprocessor 330 may temporarily transmit to the first device 100 thecontrol information allowing the use of the other user.

FIG. 9 is a view to describe an embodiment to cancel an exclusive use ofthe device according to an embodiment of the disclosure.

FIG. 9 shows a case where, when the TV 100 (e.g., the first device 100)is registered to the account of the first user A and the TV 100 is setto be exclusively used for the first user, a second user B wishes to usethe TV 100. When the second user B wishes to use the TV 100 (forexample, press a power button of a remote controller or a power buttonof TV), the TV 100 may request authentication information. In this case,the TV 100 can output a message such as “Please make the remote controlread your fingerprint for using TV” and “use is limited.” As illustratedin FIG. 9 , the message can be output from the TV 100, and the TV 100may send information informing the access of the unauthenticated user tothe second device 200 and it is possible for the second device 200 tooutput the message. For example, if the second device 200 includes aspeaker, a voice such as “the use of the TV is limited” may be output.

Thereafter, when the first user A utters a voice “let others use a TVfor one hour”, the voice is input through the microphone of the seconddevice 200, and the second device 200 may transmit the voice to theserver 300. The server 300 can analyze voice, determine that the firstuser A wishes to cancel the exclusive use of the TV 100 for an hour, andtransmit control information to the TV 100 for use of the TV 100 duringone hour.

According to still another embodiment, when the first user A does notdesignate use time and speak, for example, “let others use TV as well”,the TV 100 can be used by others during a preset time or until the TV100 is turned off.

According to still another embodiment, the first user A may designatethe second user B to use the TV 100. For example, if it is said that“let Jane (second user) use the TV”, the second user B only can use theTV for a certain period of time and the other user can use the TV 100.

According to the above-described embodiments, parental control such asrestriction of the use of children's game machines is possible. Forexample, when a child attempts to control a game machine registered withthe parent's voice, “use of this device is limited” is output. Then, ifthe parent who registered the game machine by voice says “allow use forone hour,” the limitation can be cancelled for temporary use. Inaddition, this embodiment can be used for lock/unlock a safe or apersonal storage space.

FIG. 10 is a view to describe an embodiment to request confirmation froma user to register a device according to an embodiment of thedisclosure.

FIG. 10 is a view to describe a procedure to confirm with a user toprevent a device, which a user does not wish to register, from beingregistered.

The second device 200 may acquire information about a device that theuser wants to register from the received user voice, search an externaldevice 100 (e.g., the first device 100) corresponding to the acquiredinformation, communicate with the external device 100 through acommunication connection, and transmit control information to theexternal device 100 so that information indicating that the externaldevice is to be registered is outputted from the external device 100.

The second device 200, when a plurality of external devicescorresponding to the information acquired from the voice are searched,may establish communication connection with a device having greatestsignal intensity (for example, beacon) and transmit control informationto output to the connected external device information indicating thatan external device is to be registered.

The first device 100 receiving the control information from the seconddevice 200 may output information indicating that the first device 100is ready for connection. For example, it is possible to output a voicemessage such as “ready for connection”. Such sound source files may beprestored in the device. It is also possible to visually outputinformation indicating that the device is ready for connection.

Thereafter, the second device 200 may output information for inquiringwhether to register the external device 100. For example, as shown inFIG. 10 , the second device 200 can output an inquiry voice such as “Doyou want to register the air-conditioner to be connected?” through aspeaker. If a voice (for example, “yes” “register, please”, etc.) toagree to the inquiry voice is received through the microphone 210, thesecond device 200 may transmit the information on the AP 400 and theserver authentication information to the external device 100. Then, theregistration procedure can then be performed as described in FIG. 3 .

When information inquiring whether or not the second device 200registers the external device 100 is output, and when a voice (forexample, “No,” “do not register”) not to agree to the inquiry voice isreceived, the second device 200 can terminate the registrationprocedure. Alternatively, if several devices are searched, communicationconnection can be established with a device having signal intensitywhich is second largest and have the same check process as describedabove.

According to an embodiment described above, accurate registration of adevice intended by a user can be made.

When the first device 100 is registered in the server 300, the user cancontrol the device. According to one embodiment, the user can controlthe first device 100 by voice using the second device 200. For example,when a first user who has registered the first device 100 as a voicespeaks “Turn on TV (first device)”, the second device 200 receives thevoice and transmits the voice to the server 300, and the server 300transmits a turn-on control command to the first device 100 registeredin the account of the first user, and the first device 100 receiving thecontrol command can be turned on.

According to another embodiment, it is also possible to control thefirst device 100 using a device other than the second device 200.Examples of other devices include smartphones, tablets, digital TVs(DTVs), and the like. Here, these devices are called user terminaldevices.

FIG. 11 is a view to describe a user terminal device to communicate witha server according to an embodiment of the disclosure.

Referring to FIG. 11 , a user terminal device 500 is a device registeredto the server 300 such as the first device 100 and can be used for thepurpose to control other registered devices.

An application executed in the user terminal device 500, for example, asmart home application or the like, is used to access the server 300 toregister a device, or information on the pre-registered device can beread or deleted. Control information for integrated control of thedevices can be registered or registered control information can be reador deleted.

When the smart home application is executed in the user terminal device500, at the same time as the execution of the smart home application,the user terminal device 500 may automatically log in to its own accountin the server 300 through a prestored user identifier (ID) and apassword. For some cases, by receiving the ID and password from a user,a user's account in the server 300 can be logged in. The log-ininformation is maintained for a predetermined period unless a user logsout, so within the effective period during which log-in information ismaintained, it is not necessary for a user to log in to the server 300every time.

When the first user logs in through the user terminal device 500,information to control the devices of the first user registered in theserver 300 can be transmitted to the user terminal device 500. The userterminal device 500 may configure and display a control screen based onreceived information. FIG. 11 illustrates an example of a control screendisplayed in the user terminal device 500.

The user terminal device 500 may display a control screen including alist of devices 21, 23, 25 registered to the account of the first user(Mike). A user may control the devices 21, 23, and 25. For example,power of the devices 21, 23, and 25 can be turned on or turned offthrough a control screen.

The server 300 can manage status information through periodiccommunication with devices registered in the account of the first userand can transmit status information to the user terminal device 500. Inthe user terminal device 500, status information of the devices can bedisplayed. For example, as illustrated in FIG. 11 , status informationindicating that a broadcast program of CH11 is being broadcast on the TV21, the air-conditioner 23 is operating at 23 degrees, and the robotcleaner is in the automatic cleaning mode can be displayed on the userterminal device 500.

When the information is updated in the server 300, updated informationcan also be displayed in the user terminal device 500. For example, whena new device is registered in the server 300, a new device can bedisplayed on the control screen of the user terminal device 500.According to one embodiment, the control screen displayed on the userterminal device 500 includes a menu 11 for updating information. Whenthe menu 11 is selected, the user terminal device 500 may requestinformation update to the server 300. Accordingly, when a new device isregistered by voice through the second device 200, a new device can bedisplayed also in the user terminal device 500.

According to an embodiment of the disclosure, not only a device can beregistered by voice through the second device 200, but a device can alsobe registered through the user terminal device 500. When the menu 13 forregistering a device is selected, the user terminal device 500 displaysa list of devices searched for by searching external devices in thevicinity, and when the user selects a device to be registered in thelist, the user terminal device 500 may register the selected device withthe server 300.

FIG. 12 is a flowchart to describe a controlling method of an electronicapparatus according to an embodiment of the disclosure. The flowchart onthe controlling method of the electronic apparatus of FIG. 12 can beconfigured with operations processed by the second device 200 describedin this specification. That is, the electronic apparatus can be thesecond device 200. Therefore, even if the description is omitted, thedescription relating to the second device 200 can be applied to theflowchart of FIG. 26 .

The electronic apparatus is a device which includes a microphone.Referring to FIG. 12 , the electronic apparatus may acquire informationon an external device which a user wishes to register from a voicereceived from a microphone in operation S1210.

In this case, the electronic apparatus may transmit a voice receivedthrough the microphone to the server and receive information on theexternal device from the server.

When the external device corresponding to the acquired information isfound, the electronic apparatus may transmit information on the AP tothe external device so that the external device can communicate with theserver, and transmit a registration request with respect to the externaldevice to the server in operation S1220.

In this case, the electronic apparatus may search for an external devicewhose SSID includes information on the acquired external device.

When information with respect to the plurality of devices which a userwishes to register is acquired from the voice received through themicrophone, the information on the AP can be transmitted sequentially tothe plurality of external devices and a registration request withrespect to the plurality of external devices to the server.

When information about an external device to be registered by the useris obtained from the voice, the electronic apparatus may transmitcontrol information for causing the external device to outputinformation informing that the external device is a registration targetto the external device and output voice of inquiry about theregistration of the external device through the speaker of theelectronic apparatus. The electronic apparatus may transmit informationon the access point to the external device and transmit a registrationrequest for the external device to the server when a voice agreeing withthe voice of the inquiry is received through the microphone.

In the meantime, when audio data including voice informing completion ofregistration of the external device is received from the server, theelectronic apparatus may output audio data through the speaker.

According to the aforementioned embodiments, a device registrationprocedure of a network management service in which devices are connectedvia network and managed at home and plants can be done easily using avoice, and the registered device can be controlled using the voice, thususer convenience can be improved.

The various embodiments described above may be implemented in software,hardware, or a combination thereof. In accordance with a hardwareimplementation, the embodiments described in this disclosure may beimplemented by using at least one of application specific integratedcircuits (ASICs), digital signal processors (DSPs), digital signalprocessing devices (DSPDs), programmable logic devices (PLDs), fieldprogrammable gate arrays (FPGAs), a processor, a controller, amicro-controller, a microprocessor, and an electrical unit forperforming other functions. According to a software implementation,embodiments such as the procedures and functions described herein may beimplemented in separate software modules. Each of the software modulesmay perform one or more of the functions and operations describedherein.

The various embodiments of the disclosure may be implemented as softwarethat includes instructions that may be stored in machine-readablestorage media readable by a machine (e.g., a computer). A device maycall instructions from a storage medium and that is operable inaccordance with the called instructions, including an electronicapparatus (e.g., the first device 100, the second device 200, the server300, and the user terminal device 500). When the instruction is executedby a processor, the processor may perform the function corresponding tothe instruction, either directly or under the control of the processor,using other components. The instructions may include a code generated orexecuted by the compiler or interpreter. The machine-readable storagemedium may be provided in the form of a non-transitory storage medium.Here, “non-transitory” means that the storage medium does not include asignal and is tangible, but does not distinguish whether data ispermanently or temporarily stored in a storage medium.

According to one embodiment, a method according to various embodimentsdisclosed herein may be provided in a computer program product. Acomputer program product may be traded between a seller and a purchaseras a commodity. A computer program product may be distributed in theform of a machine-readable storage medium (e.g., compact disc read onlymemory (CD-ROM)) or distributed online through an application store(e.g., PlayStore™). In the case of online distribution, at least aportion of the computer program product may be stored temporarily orgenerated temporarily in a storage medium such as a manufacturer'sserver, a server of an application store, or a memory of a relay server.

Each of the components (for example, a module or a program) according tovarious embodiments may be composed of one or a plurality of objects,and some subcomponents of the subcomponents described above may beomitted, or other subcomponents may be further included in variousembodiments. Alternatively or additionally, some components (e.g.,modules or programs) may be integrated into one entity to perform thesame or similar functions performed by each respective component priorto integration. Operations performed by a module, program, or othercomponent, in accordance with various embodiments, may be performedsequentially, in a parallel, repetitive, or heuristic manner, or atleast some operations may be performed in a different order, omitted, orother operations can be added.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. An electronic apparatus, comprising: acommunication circuit; a memory for storing instructions; and at leastone processor electrically connected to the memory, wherein the at leastone processor, upon execution of the instructions, is configured to:based on a first mode of an external device being turned on, establishcommunication with the external device, based on the communication beingestablished, control the communication circuit to transmit informationon an access point to the external device to enable the external deviceto be registered in a server, based on the external device beingregistered in the server, receive at the electronic apparatus, statusinformation of a plurality of external devices including the externaldevice and control information to control the plurality of externaldevices from the server at a time a smart home application is initiatedon the electronic apparatus or the external device is registered, andcontrol the plurality of external devices based on the controlinformation to control the external device.
 2. The electronic apparatusof claim 1, wherein the at least one processor is further configured to:register the external device in the server and control the externaldevice through the smart home application.
 3. The electronic apparatusof claim 1, further comprising a display, wherein the at least oneprocessor is further configured to: based on the external device beingregistered in the server, control the display to display a userinterface (UI) including the status information on the external device.4. The electronic apparatus of claim 3, wherein the at least oneprocessor is further configured to: control the external device throughthe UI.
 5. A method of an electronic apparatus, the method comprising:based on a first mode of an external device being turned on,establishing communication with the external device; based on thecommunication being established, transmitting information on an accesspoint to the external device to enable the external device to beregistered in a server; based on the external device being registered inthe server, receiving, at the electronic apparatus, status informationof a plurality of external devices including the external device andcontrol information to control the plurality of external devices fromthe server at a time a smart home application is initiated on theelectronic apparatus; and controlling the plurality of external devicesbased on the control information to control the external device.
 6. Themethod of claim 5, further comprising: registering the external devicein the server and control the external device through the smart homeapplication.
 7. The method of claim 5, further comprising: based on theexternal device being registered in the server, displaying a userinterface (UI) including the status information on the external device.8. The method of claim 7, further comprising: controlling the externaldevice through the UI.
 9. A non-transitory computer-readable mediumstoring thereon instructions, the instructions when executed by at leastprocessor, perform a method of an electronic apparatus, the methodcomprising: based on a first mode of an external device being turned on,establishing communication with the external device; based on thecommunication being established, transmitting information on an accesspoint to the external device to enable the external device to beregistered in a server; based on the external device being registered inthe server, receiving, at the electronic apparatus, status informationof a plurality of external devices including the external device andcontrol information to control the plurality of external devices fromthe server at a time a smart home application is initiated on theelectronic apparatus; and controlling the plurality of external devicesbased on the control information to control the external device.
 10. Thenon-transitory computer-readable medium of claim 9, the method of theelectronic apparatus further comprising: registering the external devicein the server and control the external device through the smart homeapplication.
 11. The non-transitory computer-readable medium of claim 9,the method of the electronic apparatus further comprising: based on theexternal device being registered in the server, displaying a userinterface (UI) including the status information on the external device.12. The non-transitory computer-readable medium of claim 11, the methodof the electronic apparatus further comprising: controlling the externaldevice through the UI.